Information registration device, information registration method, information registration system, information presentation device, informaton presentation method, informaton presentaton system, and program

ABSTRACT

An information registration device includes an object information acquiring unit and a content information registering unit. The object information acquiring unit acquires object information obtained by performing processing for detecting and identifying an object for image data. The content information registering unit registers target content information in a database in association with the object information.

BACKGROUND

The present technology relates to an information registration device, aninformation registration method, an information registration system, aninformation presentation device, an information presentation method, aninformation presentation system, and a program. More particularly, thepresent technology relates to an information registration device, aninformation registration method, an information registration system, aninformation presentation device, an information presentation method, aninformation presentation system, and a program that perform registrationand presentation (viewing) of content information such as images, sound,and text.

In recent years, due to the widespread use of portable informationterminals such as cellular phones, viewing of private informationincluding, for example, schedules, notes, and the like, as contentinformation, such as images, sound, and text in an electronic form, hasbeen generally performed. However, significant increases in the amountand type of data, which are caused by computerization of information,have complicated operation on information apparatuses. Thus, it hasbecome difficult to easily fetch desired information.

Meanwhile, due to the development of text-based search methods, byappropriately adding meta-information (a title, a tag, etc.) toinformation, it has become very easy to fetch desired information on thebasis of keywords. However, when different types of meta-information areadded to a piece of information, it is difficult to recall appropriatekeywords to be used for searching. To this end, user interfaces forregistration and viewing of information are to be improved.

Technology for overlaying information on camera-through images using anobject recognizing technique is suggested, for example, in JapanesePatent No. 3558104.

SUMMARY

It is desirable to improve the user-friendliness of registration andpresentation (viewing) of content information such as images, sound, andtext.

A concept of an embodiment of the present technology lies in aninformation registration device including an object informationacquiring unit that acquires object information obtained by performingprocessing for detecting and identifying an object for image data, and acontent information registering unit that registers target contentinformation in a database in association with the object information.

In an embodiment of the present technology, the object informationacquiring unit acquires object information obtained by performingprocessing for detecting and identifying an object for image data. Forexample, the object information may include meta-information on theobject as well as object identification information. Themeta-information is, for example, at least one of positional informationon the object and attitude information on the object. With asimultaneously localization and mapping (SLAM) technique, an imagetracking technique, the value obtained by a gyro sensor, or the like,the above-mentioned information can be corrected and prediction ofinformation can be performed in an undetected state after an object isdetected.

In this case, acquiring object information by performing processing fordetecting and identifying an object for image data may be performedexternally, for example, by a server device that is connected over anetwork or performed by an object recognizing unit provided in theinformation registration device. Furthermore, in this case, the imagedata may be image data stored in a storing unit, such as a semiconductormemory or a hard disk drive, or image data obtained by an image pickupunit (camera) provided in the information registration device.

The content information registering unit registers target contentinformation in the database in association with the object information.In this case, the target content information is content information suchas images, sound, and text. In this case, the database may be provided,for example, in the server device that is connected over the network orin the information registration device.

As described above, in an embodiment of the present technology, targetcontent information is registered in the database in association withobject information. Thus, a user does not have to consider appropriatesearch keywords in order to register content information, thususer-friendliness being improved. That is, a user is able to registercontent information only by performing an operation for capturing animage of a specific object or an operation for selecting image dataobtained by capturing a specific object and stored in a storing unit.

Furthermore, in an embodiment of the present technology, for example,the information registration device may further include a useridentification information acquiring unit that acquires useidentification information obtained by processing for detecting andidentifying a user. The content information registering unit mayregister target content information in the database in association withobject information and the user identification information.

In this case, acquiring the user identification information by theprocessing for detecting and identifying a user may be performedexternally, for example, by a server device that is connected over anetwork or performed by a user recognizing unit provide in theinformation registration device. The processing for detecting andidentifying a user may be performed on the basis of image recognition,such as face recognition, or fingerprint recognition, or on the basis ofa user ID and a password manually entered by a user. That is, theprocessing for detecting and identifying a user may be generalprocessing of user authentication.

In this case, target content information is registered in the databasein association with the user identification information as well as theobject information. Thus, it is easy to fetch desired contentinformation associated with the user identification information as wellas the object information, from the database. That is, contentinformation suitable for a user can be fetched.

Furthermore, in an embodiment of the present technology, for example,the information registration device may further include a registrationenvironmental information acquiring unit that acquires registrationenvironmental information. The content information registering unit mayregister target content information in the database in association withobject information and the registration environmental information. Forexample, the registration environmental information may include at leastone of positional information and time information.

In this case, the registration environmental information may be acquiredusing a clock unit, a positional information acquiring unit, and thelike or may be, for example, externally acquired via a network. Theregistration environmental information may include weather information,climate information, and the like as well as information acquired by thepositional information acquiring unit (latitude and longitude,elevation, velocity, moving direction, etc.) and time informationacquired by the clock unit. The registration environmental informationmay further include user information on a user who can view information,that is, user information on a user who is permitted to viewinformation.

Furthermore, the registration environmental information may also includepart or all of the following information: for example, (1) temperature,atmospheric pressure, and humidity, (2) orientation and acceleration,(3) proximity, (4) illuminance, (5) noise, (6) illuminance, (7) noise,(8) date, (9) attitude (six axes), and (10) text. Furthermore, theregistration environmental information may include, for example, (11)user's vital data (complexion, body temperature, body mass index (BMI),blood pressure, heart rate, fat mass, basal metabolism, fingerprints,voiceprints, etc.), (12) user's operation (commuting, playing golf,having dinner, cleaning, etc., which may be automatically detected by asensor or acquired in accordance with a manual operation by a user),(13) sound (may be automatically detected by a sensor or acquired byuser's recording), and (14) information viewing history.

In this case, target content information is registered in the databasein association with the registration environmental information as wellas the object information. Thus, on the basis of the registrationenvironmental information, content information suitable for a viewingenvironment can be preferentially presented.

Furthermore, another concept of the present technology lies in aninformation presentation device including an object informationacquiring unit that acquires object information obtained by performingprocessing for detecting and identifying an object for image data, and acontent information presenting unit that acquires content informationassociated with the object information and presents the acquired contentinformation.

In an embodiment of the present technology, the object informationacquiring unit acquires object information obtained by performingprocessing for detecting and identifying an object for image data. Forexample, the object information may include meta-information on theobject as well as object identification information. Themeta-information is, for example, at least one of positional informationon the object and attitude information on the object. With a SLAMtechnique, an image tracking technique, the value obtained by a gyrosensor, or the like, the above-mentioned information can be correctedand prediction of information can be performed in an undetected stateafter an object is detected.

In this case, acquiring object information by performing processing fordetecting and identifying an object for image data may be performedexternally, for example, by a server device that is connected over anetwork or performed by an object recognizing unit provided in theinformation presentation device. Furthermore, in this case, the imagedata may be image data stored in a storing unit, such as a semiconductormemory or a hard disk drive, or image data obtained by an image pickupunit (camera) provided in the information presentation device.

The content information presenting unit acquires content informationassociated with the object information and presents the acquired contentinformation. The content information is content information such asimages, sound, and text. The content information is, for example,content information registered in advance in a database by a user orcontent information acquired via a network such as the Internet.Furthermore, in this case, the database may be provided, for example, ina server device that is connected via a network or in the informationpresentation device.

As described above, in an embodiment of the present technology, contentinformation associated with the object information is acquired andpresented. Thus, a user is able to view content information withoutperforming a complicated operation, thus user-friendliness beingimproved. That is, desired content information can be presented, and auser is able to view the presented information only by performing anoperation for capturing an image of a specific object or performing anoperation for selecting image data obtained by capturing a specificobject and stored in a storing unit.

In an embodiment of the present technology, the information presentationdevice may further include a user identification information acquiringunit that acquires user identification information obtained byprocessing for detecting and identifying a user. The content informationpresenting unit may acquire content information associated with objectinformation and the user identification information and present theacquired content information.

In this case, acquiring the user identification information by theprocessing for detecting and identifying a user may be performedexternally, for example, by a server device that is connected over anetwork or performed by a user recognizing unit provided in theinformation presentation device. The processing for detecting andidentifying a user may be performed on the basis of image recognition,such as face recognition, or fingerprint recognition, or on the basis ofa user ID and a password manually entered by a user. That is, theprocessing for detecting and identifying a user may be generalprocessing of user authentication.

In this case, content information associated with object information anduser identification information is acquired. That is, desired contentinformation associated with the user identification information as wellas the object information can be acquired. That is, content informationsuitable for a user can be fetched.

Furthermore, in an embodiment of the present technology, the informationpresentation device may further include a viewing environmentalinformation acquiring unit that acquires viewing environmentalinformation. The content information acquired by the content informationpresenting unit may include registration environmental information addedthereto. The content information presenting unit may determine the orderin which a specific number of pieces of acquired content information isto be presented, on the basis of the viewing environmental informationand the registration environmental information.

In this case, the viewing environmental information may be acquiredusing a clock unit, a positional information acquiring unit, and thelike or may be externally acquired via a network. The viewingenvironmental information may include weather information, climateinformation, and the like as well as information acquired by thepositional information acquiring unit (latitude and longitude,elevation, velocity, moving direction, etc.) and time informationacquired by the clock unit.

Furthermore, the viewing environmental information may also include partor all of the following information: for example, (1) temperature,atmospheric pressure, and humidity, (2) orientation and acceleration,(3) proximity, (4) illuminance, (5) noise, (6) illuminance, (7) noise,(8) date, (9) attitude (six axes), and (10) text. Furthermore, theviewing environmental information may include, for example, (11) user'svital data (complexion, body temperature, BMI, blood pressure, heartrate, fat mass, basal metabolism, fingerprints, voiceprints, etc.), (12)user's operation (commuting, playing golf, having dinner, cleaning,etc., which may be automatically detected by a sensor or acquired inaccordance with a manual operation by a user), (13) sound (may beautomatically detected by a sensor or acquired by user's recording), and(14) information viewing history.

In this case, the order in which a specific number of pieces of acquiredcontent information are to be presented is determined on the basis ofthe viewing environmental information. That is, content informationsuitable for a viewing environment can be preferentially presented.

Furthermore, in an embodiment of the present technology, for example,the information presentation device may further include, for example, animage pickup unit that acquires image data, and a relative positionalinformation acquiring unit that acquires relative positional informationon the relative position between an object and the image pickup unit,the relative positional information being obtained by performingprocessing on the image data. The content information presenting unitmay acquire content information associated with object information andthe relative positional information and presents the acquired contentinformation.

In this case, content information associated with object information andrelative positional information is acquired. That is, desired contentinformation associate with an operation, such as, for example, forming aspecific figure, performed with the image pickup unit (camera) as wellas object information can be acquired.

According to an embodiment of the present technology, theuser-friendliness of registration and presentation (viewing) of contentinformation such as images, sound, and text can be improved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of the configurationof an information registration system according to a first embodiment ofthe present technology;

FIG. 2A is a diagram for explaining that a system includes only a clientdevice;

FIG. 2B is a diagram for explaining that the system is a network systemincluding the client device and a server device that are connected overa network;

FIG. 3 is a flowchart illustrating an example of a process ofinformation registration in the information registration system;

FIG. 4 is a block diagram illustrating an example of the configurationof an information presentation system according to a second embodimentof the present technology;

FIG. 5 is a flowchart illustrating an example of a process ofinformation presentation in the information presentation system;

FIG. 6 includes diagrams for explaining an example of informationviewing at the time of visiting a destination as a specific example;

FIG. 7 includes diagrams for explaining an example of informationviewing at the time of visiting a destination as a specific example;

FIG. 8 is a block diagram illustrating another example of theconfiguration of an information presentation system;

FIG. 9 is a diagram for explaining an example in which information onthe position (x, y, z) and the rotation (roll, pitch, yaw) of an objectas well as object identification information is used; and

FIG. 10 includes diagrams for explaining an example in which informationon the position (x, y, z) and the rotation (roll, pitch, yaw) of anobject as well as object identification information is used.

FIG. 11 is a block diagram illustrating an example of the configurationof an information presentation system according to a third embodiment ofthe present technology;

FIG. 12 is a flowchart illustrating an example of a process ofinformation presentation in the information presentation system;

FIG. 13 is a diagram for explaining gesture recognition based onposition;

FIG. 14 includes diagrams for explaining an example of an informationviewing system as a specific example;

FIG. 15 includes diagrams for explaining an example of an informationviewing system as a specific example;

FIG. 16 is a diagram for explaining an example of an item informationpresentation system as a specific example;

FIG. 17 is a block diagram illustrating an example of the configurationof an information registration system; and

FIG. 18 is a flowchart illustrating an example of a process ofinformation registration in the information registration system.

DETAILED DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments implementing the present technology(hereinafter, referred to as “embodiments”) will be described. Thedescription will be given in the following order:

1. First Embodiment

2. Second Embodiment

3. Third Embodiment

4. Modifications

1. First Embodiment Information Registration System

FIG. 1 illustrates an example of the configuration of an informationregistration system 100 according to a first embodiment of the presenttechnology. The information registration system 100 includes an objectrecognizing unit 101, an object identification database 102, a userrecognizing unit 103, a user identification database 104, and aregistration environment detecting unit 105. The informationregistration system 100 also includes an information registering unit106, an information input unit 107, a presenting unit 108, an operationinput unit 109, and a registration information database 110.

The object recognizing unit 101 performs processing for detecting andidentifying an object for image data and obtains object information, onthe basis of object identification data stored in the objectidentification database 102. The image data used here may be image datastored in a semiconductor memory, a hard disk drive, or the like servingas a storing unit or image data obtained by capturing an image with animage pickup unit (camera). In this case, the common objectidentification database 102 is used in a system so that an object is notrecognized as a different object. Here, objects include human beings andanimals.

An object state obtained by the object recognizing unit 101 contains,for example, object identification information (object ID) andinformation on the position and attitude of an object asmeta-information on the object. With a simultaneously localization andmapping (SLAM) technique, an image tracking technique, the valueobtained by a gyro sensor, or the like, the above-mentioned informationcan be corrected and prediction of information can be performed in anundetected state after an object is detected. The object state acquiredby the object recognizing unit 101 may include information on theposition or the attitude. For example, the object recognizing unit 101recognizes an object on the basis of the positions of characteristicpoints of an object contained in an image. Furthermore, the objectrecognizing unit 101 calculates the position and attitude of an objectin a world coordinate system, on the basis of the positions ofcharacteristic points of the object. This positional informationincludes relative positional information on the relative positionbetween an object and an image pickup unit (camera). The processingperformed by the object recognizing unit 101 will not be explained indetail below and is described, for example, in Japanese UnexaminedPatent Application Publication No. 2008-304268.

The user recognizing unit 103 performs processing for detecting andidentifying a user and obtains user identification information, on thebasis of user identification data stored in the user identificationdatabase 104. The processing for detecting and identifying a user may beperformed on the basis of image recognition, such as face recognition,or fingerprint recognition, or on the basis of a user ID and a passwordmanually entered by a user. That is, the processing for detecting andidentifying a user may be general processing of user authentication. Inthis case, the common user identification database 104 is used in asystem so that a user (human being) is not recognized as a differentuser.

The registration environment detecting unit 105 detects environmentalinformation at the time of registration, from an informationregistration terminal, the Internet, or the like, which is notillustrated. The environmental information includes, for example,information (latitude and longitude, elevation, velocity, movingdirection, etc.) acquired by a positional information acquiring unit,time information acquired by a clock unit, weather information acquiredfrom the Internet, and the like. In this case, the common methods foradjusting time and calculating latitude and longitude are used in asystem so that an environment is not recognized as a differentenvironment. The environmental information to be registered may includeuser information on a user who can view information.

The positional information acquiring unit acquires information (latitudeand longitude, elevation, velocity, moving direction, etc.) using aglobal positioning system (GPS) or wireless fidelity (WiFi) measurementor measurement using a portable base station.

Environmental information may also include part or all of the followinginformation: for example, (1) temperature, atmospheric pressure, andhumidity, (2) orientation and acceleration, (3) proximity, (4)illuminance, (5) noise, (6) illuminance, (7) noise, (8) date, (9)attitude (six axes), and (10) text. Furthermore, environmentalinformation may include, for example, (11) user's vital data(complexion, body temperature, body mass index (BMI), blood pressure,heart rate, fat mass, basal metabolism, fingerprints, voiceprints,etc.), (12) user's operation (commuting, playing golf, having dinner,cleaning, etc., which may be automatically detected by a sensor oracquired in accordance with a manual operation by a user), (13) sound(may be automatically detected by a sensor or acquired by user'srecording), and (14) information viewing history.

The information registering unit 106 registers content information inputfrom the information input unit 107 in the registration informationdatabase 110, in association with the object information, the useridentification information, and the registration environmentalinformation mentioned above. The content information is, for example,various types of content information such as images, sound, and text.The information registering unit 106 registers content information ofthe type selected by the user, in accordance with an operation by theuser with the operation input unit 109 using a graphical user interface(GUI) displayed on the presenting unit 108.

In this case, the user may be able to select a type of environmentalinformation that is to be linked with content information to beregistered. For example, in the case where positional information, timeinformation, and weather information are acquired by the registrationenvironment detecting unit 105, only the positional information and thetime information can be selected as environmental information to belinked with content information to be registered.

The information registration system 100 illustrated in FIG. 1 can be,for example, a client device 500A illustrated in FIG. 2A or a networksystem illustrated in FIG. 2B including the client device 500A and aserver device 500B that are connected over a network 500C. In the caseof the network system illustrated in FIG. 2B, units forming theinformation registration system 100 are appropriately allocated in adistributed manner to the client device 500A and the server device 500B.For example, the object recognizing unit 101, the object identificationdatabase 102, the user recognizing unit 103, the user identificationdatabase 104, part of the registration environment detecting unit 105,and the registration information database 110 can be allocated to theserver device 500B.

An example of a process for performing information registration by theinformation registration system 100 illustrated in FIG. 1 will now beexplained with reference to a flowchart illustrated in FIG. 3. In stepST1, the information registration system 100 starts the process. Then,the information registration system 100 proceeds to processing of stepST2. In step ST2, the user recognizing unit 103 performs processing fordetecting and identifying a user and obtains user identificationinformation, on the basis of input user information (a face image, afingerprint image, manually input information on an account and apassword of the user, etc.).

In step ST3, the object recognizing unit 101 performs processing fordetecting and identifying an object for input image data, and obtainsobject information. In this processing, for example, the user capturesan image of a specific object with an image pickup unit (camera)attached to the client device 500A and obtains input image data.Alternatively, for example, the user selects image data stored in astoring unit of the client device 500A and obtains input image data.

In step ST4, the presenting unit 108 presents the target (object)identified in step ST3 for which information can be registered. In stepST5, the information registration system 100 determines whether or notan operation for selecting the target presented in step ST4 has beenperformed by the user. The user is able to perform an operation forselection by operating the operation input unit 109.

When an operation for selection has been performed by the user, theinformation registration system 100 proceeds to step ST6. In step ST6,the presenting unit 108 presents a type of content information that canbe input. For example, in the case where any of images, sound, and textcan be input, the presenting unit 108 presents a message indicating thatany of images, sound, and text can be input. In step ST7, theinformation registration system 100 determines whether or not thepresented type of content information has been input. The user is ableto input desired content information using the information input unit107.

When content information is input, the presenting unit 108 presents aregistration environment detected by the registration environmentdetecting unit 105 in step ST8. In step ST9, the informationregistration system 100 determines whether or not an operation forselecting the presented environment has been performed. The user is ableto perform an operation for selection by operating the operation inputunit 109.

When an operation for selection has been performed by the user, theinformation registration system 100 proceeds to processing of step ST10.In step ST10, the information registering unit 106 registers the contentinformation input in step ST7 for the target (object) selected in stepST5. That is, the information registering unit 106 registers the contentinformation in the registration information database 110, in associationwith the target (object). In this processing, the content information isalso associated with the user identification information and theregistration environmental information selected in step ST9. After theprocessing of step ST10, the information registration system 100terminates the process in step ST11.

As described above, in the information registration system 100illustrated in FIG. 1, target content information is registered in theregistration information database 110, in association with objectinformation. Thus, the user is able to register content informationwithout considering an appropriate search keyword, thususer-friendliness being improved. That is, the user is able to registercontent information only by performing an operation for capturing animage of a specific object or performing an operation for selectingimage data stored in a storing unit and obtained by capturing an imageof a specific object.

Furthermore, in the information registration system 100 illustrated inFIG. 1, target content information is registered in the registrationinformation database 110, in association with the user identificationinformation as well as the object information. Thus, it is easy to fetchdesired content information associated with user identificationinformation as well as object information, from the registrationinformation database 110. That is, content information suitable for auser can be fetched.

Furthermore, in the information registration system 100 illustrated inFIG. 1, target content information is registered in the registrationinformation database 110, in association with registration environmentalinformation as well as object information. Thus, on the basis of theregistration environmental information, content information suitable fora viewing environment can be preferentially presented.

2. Second Embodiment Information Presentation System

FIG. 4 illustrates an example of the configuration of an informationpresentation system 200 according to a second embodiment of the presenttechnology. The information presentation system 200 includes an objectrecognizing unit 201, an object identification database 202, a userrecognizing unit 203, a user identification database 204, and a viewingenvironment detecting unit 205. The information presentation system 200also includes a user state detecting unit 206, an information presentingunit 207, a registration information database 208, the Internet 209, andan operation input unit 210.

The object recognizing unit 201 performs processing for detecting andidentifying an object for image data and obtains object information, onthe basis of object identification data stored in the objectidentification database 202. The image data used here may be image datastored in a semiconductor memory, a hard disk drive, or the like servingas a storing unit or image data obtained by capturing an image with animage pickup unit (camera). In this case, the common objectidentification database 202 is used in a system so that an object is notrecognized as a different object. Here, objects include human beings andanimals.

An object state obtained by the object recognizing unit 201 contains,for example, object identification information (object ID) andinformation on the position and attitude of an object asmeta-information on the object. With a SLAM technique, an image trackingtechnique, the value obtained by a gyro sensor, or the like, theabove-mentioned information can be corrected and prediction ofinformation can be performed in an undetected state after an object isdetected. The object state acquired by the object recognizing unit 201may be information on the position or the attitude. For example, theobject recognizing unit 201 recognizes an object on the basis of thepositions of characteristic points of an object contained in an image.Furthermore, the object recognizing unit 201 calculates the position andattitude of an object in a world coordinate system, on the basis of thepositions of characteristic points of the object. This positionalinformation includes relative positional information on the relativeposition between an object and an image pickup unit (camera). Theprocessing performed by the object recognizing unit 201 will not beexplained in detail below and is described, for example, in JapaneseUnexamined Patent Application Publication No. 2008-304268.

The user recognizing unit 203 performs processing for detecting andidentifying a user and obtains user identification information, on thebasis of user identification data stored in the user identificationdatabase 204. The processing for detecting and identifying a user may beperformed on the basis of image recognition, such as face recognition,or fingerprint recognition, or on the basis of a user ID and a passwordmanually entered by a user. That is, the processing for detecting andidentifying a user may be general processing of user authentication. Inthis case, the common user identification database 204 is used in asystem so that a user (human being) is not recognized as a differentuser.

The viewing environment detecting unit 205 detects environmentalinformation at the time of viewing, from an information registrationterminal, the Internet, or the like, which is not illustrated. Theenvironmental information includes, for example, information (latitudeand longitude, elevation, velocity, moving direction, etc.) acquired bya positional information acquiring unit, time information acquired by aclock unit, information acquired from the Internet, and the like. Inthis case, the common methods for adjusting time and calculatinglatitude and longitude are used in a system so that an environment isnot recognized as a different environment.

The positional information acquiring unit acquires information (latitudeand longitude, elevation, velocity, moving direction, etc.) using a GPSor WiFi measurement or measurement using a portable base station.

Environmental information may also include part or all of the followinginformation: for example, (1) temperature, atmospheric pressure, andhumidity, (2) orientation and acceleration, (3) proximity, (4)illuminance, (5) noise, (6) illuminance, (7) noise, (8) date, (9)attitude (six axes), and (10) text. Furthermore, environmentalinformation may include, for example, (11) user's vital data(complexion, body temperature, BMI, blood pressure, heart rate, fatmass, basal metabolism, fingerprints, voiceprints, etc.), (12) user'soperation (commuting, playing golf, having dinner, cleaning, etc., whichmay be automatically detected by a sensor or acquired in accordance witha manual operation by a user), (13) sound (may be automatically detectedby a sensor or acquired by user's recording), and (14) informationviewing history.

The user state detecting unit 206 generates an information search query,on the basis of object identification information, user identificationinformation, viewing environmental information, and the like. Forexample, the information search query is generated in order to presentcontent information associated with object identification informationand user identification information included in object information, inthe order based on viewing environmental information (time information,positional information, etc.).

The information presenting unit 207 acquires content information fromthe registration information database 208, on the basis of theinformation search query generated by the user state detecting unit 206.The information presenting unit 207 also acquires associated informationfrom the Internet 209 and presents the acquired information. In thiscase, when a plurality of pieces of information are acquired,information to be presented is switched automatically in accordance withthe priority of information or by a switching operation by a user usingthe operation input unit 210. In this case, for example, the operationinput unit 210 may include a touch sensor or an acceleration sensor.Associated information may not be acquired from the Internet 209.

The information presentation system 200 illustrated in FIG. 4 can alsobe, for example, the client device 500A illustrated in FIG. 2A or thenetwork system illustrated in FIG. 2B including the client device 500Aand the server device 500B that are connected over the network 500C. Inthe case of the network system illustrated in FIG. 2B, units forming theinformation presentation system 200 are appropriately allocated in adistributed manner to the client device 500A and the server device 500B.For example, the object recognizing unit 201, the object identificationdatabase 202, the user recognizing unit 203, the user identificationdatabase 204, part of the viewing environment detecting unit 205, andthe registration information database 208 can be allocated to the serverdevice 500B.

An example of a process for performing information presentation by theinformation presentation system 200 illustrated in FIG. 4 will now beexplained with reference to a flowchart illustrated in FIG. 5. In stepST21, the information presentation system 200 starts the process, forexample, when the power is turned on. Then, the information presentationsystem 200 proceeds to processing of step ST22. In step ST22, the userrecognizing unit 203 performs processing for detecting and identifying auser and obtains user identification information, on the basis of inputuser information (a face image, a fingerprint image, manually inputinformation on an account and a password of the user, etc.).

In step ST23, the viewing environment detecting unit 205 detectsenvironmental information at the time of viewing, such as positionalinformation (latitude and longitude information), time information, andthe like, from an information registration terminal, the Internet, orthe like, which is not illustrated. Then, the information presentationsystem 200 proceeds to step ST24.

In step ST24, the object recognizing unit 201 performs processing fordetecting and identifying an object for input image data, and obtainsobject information. In this processing, for example, the user capturesan image of a specific object with an image pickup unit (camera)attached to the client device 500A and obtains input image data.Alternatively, for example, the user selects image data stored in astoring unit of the client device 500A and obtains input image data.

In step ST25, the information presentation system 200 determines whetheror not an object has been identified. When an object has beenidentified, in step ST26, the user state detecting unit 206 generates aninformation search query, on the basis of the object identificationinformation, the user identification information, the viewingenvironmental information, and the like. For example, the informationsearch query is generated in order to present content informationassociated with object identification information and useridentification information included in object information, in accordancewith the order based on viewing environmental information (timeinformation, positional information, etc.).

In step ST27, the information presenting unit 207 acquires informationfrom the registration information database 208 and the Internet 209 onthe basis of the information search query generated in step ST26, andpresents the acquired information. In this case, content informationregistered in association with the object identification information andthe user identification information is acquired from the registrationinformation database 208. Here, for example, even if user identificationinformation does not match the information search query, when the useridentification information is within an allowable range (for example,relatives, friends, etc.) for user information on a user who can viewinformation included in environmental information at the time ofregistration, corresponding content information is acquired.Furthermore, in this case, associated information is acquired from theInternet 209.

In step ST28, the information presentation system 200 determines whetheror not an operation for switching has been performed. An operation forswitching is, for example, performed in accordance with a switchingoperation performed by the user using the operation input unit 210 orperformed automatically after a certain period of time has passed. Whenan operation for switching has been performed, the informationpresenting unit 207 acquires information in the next place in thepresentation order from the registration information database 208 andthe Internet 209 and presents the acquired information in step ST27.When only one piece of information is to be presented, the informationis continued to be presented. When an operation for switching isperformed while the information in the last place in the presentationorder of a plurality of pieces of information to be presented is beingpresented, switching to presentation of the information in the firstplace in the presentation order is performed.

When an operation for switching has not been performed in step ST28, theinformation presentation system 200 determines whether or not the objectidentification state where the object is identified, the state beingentered in step ST25, is maintained in step ST29. When the objectidentification state is maintained, the information presentation system200 returns to the processing of step ST28. When the objectidentification state is not maintained, the information presentationsystem 200 returns to the processing of step ST24, and repeatsprocessing similar to the processing described above.

For example, when a user cancels the state where an image of an objectis being captured and the system enters the state where an image of adifferent object is being captured, the object identification state isinterrupted. Thus, the process returns to the processing of step ST24.When the object identification state is not maintained, the process maynot return to the processing of step ST24 until the user performs anexplicit operation. The information presentation system 200 terminatesthe process, for example, when the power is turned off.

SPECIFIC EXAMPLES

Specific examples will now be described. Here, an example of informationviewing at the time of visiting a destination, an example of areputation viewing system, and an example of a bulletin board systemwill be explained as a first example, a second example, and a thirdexample, respectively.

First Example Information Viewing at the Time of Visiting a Destination

When visiting a destination, a user wants to quickly acquire desiredinformation. In such a situation, a system in which “attachment ofinformation” and “fetching of information” can be performed for anobject (item) that is carried with a user when the user visits somewhereis implemented, using the information registration system 100illustrated in FIG. 1 and the information presentation system 200illustrated in FIG. 4.

In this example, an information terminal such as a cellular phone thatis capable of accessing the Internet and that includes a camera (imagepickup unit) is assumed. After the user holds the information terminalover a target and captures an image of the object in advance asillustrated in part (a) of FIG. 6, information on a visit is registered,as illustrated in part (b) of FIG. 6. Then, when the user holds theinformation terminal over the target as illustrated in part (a) of FIG.7, information is presented, as illustrated in parts (b) to (d) of FIG.7.

At this time, information search is performed for the registrationinformation database 208 on the basis of identification results obtainedby the object recognizing unit 201 and the user recognizing unit 203.Furthermore, not only is registered information displayed, informationto be used for the visit is also acquired from the Internet andpresented. Information to be presented is switched in accordance with auser operation through a touch sensor or an acceleration sensor or aftera certain period of time has passed.

In addition, the order of presentation is controlled on the basis ofordering of information performed by the user state detecting unit 206.For example, the ordering is done on the basis of the prioritycalculated for each piece of information, using equation (1):V=a ₀ ·D ₀(t)+a ₁ ·D ₁(x,y)  (1),

where D₀(t) represents the time to a scheduled time, D₁(x, y) representsthe distance to a destination, and a₀ and a₁ each represent acoefficient.

In equation (1), “the time to a scheduled time” and “the distance to adestination” are normalized on the basis of viewing environment at thetime of presentation of information, and the priority V is calculated.In a display example illustrated in FIG. 7, on the basis of the priorityV, “Schedule”, “Movement Information”, and “Weather Information” aredisplayed in that order. Here, information can be attached to variousobjects (items) such as a golf membership card to be used when going toa golf driving range, a patient's registration card to be used whenseeing a doctor, a staff identity card to be used when going to office,and the like.

Second Example Reputation Viewing System

When considering whether to buy confectionery, a toy, or the like, auser wants to easily register and view reputations for a target. In sucha situation, a system in which “attaching of information” and “fetchingof information” can be performed for a target is implemented, using theinformation registration system 100 illustrated in FIG. 1 and theinformation presentation system 200 illustrated in FIG. 4.

In this example, an information terminal such as a cellular phone thatis capable of accessing the Internet and that includes a camera (imagepickup unit) is assumed. Basically, the same system as that used in thefirst example is implemented. The system used in the second example isdifferent from the system used in the first example in behavior of theuser state detecting unit 206. In the first example, a useridentification result is used only when information search is performed.In the second example, however, a user identification result is used forcalculating the priority V, as in equation (2):V=a ₀ ·D ₀(t)+a ₁ ·D ₁(x,y)+a ₂ ·D ₂(n)  (2),

where “D₀(t)” represents the time to the information registration time,“D₁(x, y)” represents the distance to the information registrationplace, “D₂(n)” represents the distance between users who registerinformation, and “a₀”, “a₁”, and “a₂” each represent a coefficient.

For example, the weighting sum of differences in a value, such as age,sex, annual income, BMI, or the like, is normalized. Accordingly, a useris able to easily view information registered by various people inaccordance with the importance, only by holding the information terminalover a target. This example can be applied not only to purchase ofconfectionery but also to various cases where selection of any item isperformed. Furthermore, the second example is also applied to a casewhere an item seller or the like registers campaign information or thelike. In the case of Twitter or the like, time line display or the likebased on the relationship among people, such as followers, is available.In this example, however, information is presented on the basis of anobject.

Third Example Bulletin Board System

In the situation of getting together in a large group, users want toleave messages in a particular place. Message boards at stations havedrawbacks in that, for example, the general public see messages, messageboards are available only limited places, and it is difficult for a userto discriminate information related to the user from a large volume ofinformation provided by many people. In such a situation, a bulletinboard system that can solve the above-mentioned drawbacks and that canbe installed in a desired place is implemented, using the informationregistration system 100 illustrated in FIG. 1 and the informationpresentation system 200 illustrated in FIG. 4.

This bulletin board system can be accessed from a normal web browser.After logging in, a user selects a mark registered in advance in asystem, the mark being presented together with a place where the mark islocated, and selects and registers target information together with therelative position and rotation information for the mark. At this time,restriction on viewing of the information can also be set.

For restriction on viewing, setting may be performed such thatregistered information can be viewed only when a gesture of forming atriangle is performed in front of the mark. Accordingly, filtering ofinformation can be performed, using the gesture, for a large volume ofinformation registered for the mark by the user or a third party.

With this system, for example, since information can be registered whena user stays at the user's home, the user can use this system in such amanner that the user selects the loyal dog Hachi-Ko in front of ShibuyaStation as a meeting place for a drinking party and registers asinformation, in a space on the slightly right of a bronze plate with thewords “the loyal dog Hachi-Ko”, the place where the drinking party isheld and the map of the place.

This bulletin board system can be accessed from an information terminalsuch as a cellular phone including a camera (image pickup unit). Thatis, by capturing an image of a mark registered in advance in the system,a user is able to view information registered for the mark. At thistime, information that is suitable for the current state of the user ispreferentially presented from among various types of informationregistered in association with the state of the user. Furthermore, byperforming a gesture of forming a particular shape, such as a triangleor a pentagram, in front of a mark for presented information, switching,filtering, and the like of information can be performed.

Accordingly, for example, in the case where a user registers informationfor a bronze plate in front of the loyal dog Hachi-Ko when the user isat home, by actually visiting the site, capturing an image of the bronzeplate with a camera, and performing a gesture of forming a triangle, theuser is able to view information. In this case, the user is able to viewinformation when the user knows which gesture the user should perform infront of a mark. Thus, people who are supposed to get together canaccess the information if they know the mark and the gesture in advance.If a person who does not know the mark and the gesture, it is difficultfor the person to access the information. Accordingly, a function ofenabling only a desired person to view information can be easilyimplemented.

In addition, in this bulletin board system, information can beregistered using a portable terminal. That is, after capturing an imageof a mark registered in advance in the system, by calling an informationregistration GUI and performing operation on the informationregistration GUI, a user is able to register information. Thus, forexample, in the case of registering information for the above-mentionedmeeting place, the user is able to register as a note, on the lowerright or the like of the bronze plate with the words “the loyal dogHachi-Ko”, information indicating that the user has already arrived atthe meeting place and the user is going ahead first to the drinkingparty site.

An information presentation system 200A illustrated in FIG. 8 includes agesture recognizing unit 211, so as to support the third exampledescribed above. In FIG. 8, parts corresponding to those in FIG. 4 arereferred to with the same reference numerals and the description ofthose parts will be omitted.

The gesture recognizing unit 211 performs processing for detecting andidentifying a gesture for positional information of an object outputfrom the object recognizing unit 201, on the basis of gestureidentification data stored in a gesture identification database 212, andobtains gesture identification information. Here, object positionalinformation includes relative positional information on the relativeposition between an object and a camera (image pickup unit).

The information presenting unit 207 acquires information from theregistration information database 208 and the Internet 209, on the basisof an information search query and gesture identification informationfrom the user state detecting unit 206, and presents the acquiredinformation. Although not explained in detail, the other parts of theinformation presentation system 200A are configured similarly to thoseof the information presentation system 200 illustrated in FIG. 4.

In the description provided above, content information such as images,sound, and text is registered in the registration information database208, in association with object identification information. Theinformation presenting unit 207 acquires, from the registrationinformation database 208, content information associated with objectidentification information, and displays the acquired contentinformation.

However, for example, a user may want to register content information inthe registration information database 208, in association with theposition (x, y, z) and the rotation (roll, pitch, yaw) for an object, aswell as object identification information. With the informationregistration system 100 illustrated in FIG. 1, such information can beregistered. FIG. 9 illustrates that message information is registeredfor a position (x1, y1, z1), which is away from the position where anobject is located.

In this case, in the information presentation system 200 (for example, aportable terminal), when the image pickup position is moved from aposition illustrated in part (a) of FIG. 10 to a position illustrated inpart (b) of FIG. 10, an object is moved out of the frame of a camera, asillustrated in part (c) to (d) of FIG. 10. With this operation, messageinformation associated with the position (x1, y1, z1) can be acquiredfrom the registration information database 208, as described above.Accordingly, hidden message information can be displayed, as illustratedin part (d) of FIG. 10.

As described above, in the information presentation system 200illustrated in FIG. 4, content information associated with objectinformation (object identification information) is acquired from theregistration information database 208, and the acquired contentinformation is presented. Thus, a user is able to view contentinformation without performing a complicated operation, thususer-friendliness being improved. That is, desired content informationcan be presented to the user, and the user is able to view the presentedinformation only by performing an operation for capturing an image of aspecific object or selecting image data obtained by capturing an imageof a specific object and stored in a storing unit.

Furthermore, in the information presentation system 200 illustrated inFIG. 4, content information associated with object information and useridentification information can be acquired from the registrationinformation database 208. That is, desired content informationassociated with user identification information as well as objectinformation can be acquired. Namely, content information suitable for auser can be fetched.

Furthermore, in the information presentation system 200 illustrated inFIG. 4, the order in which a specific number of pieces of acquiredcontent information is to be presented can be determined on the basis ofviewing environmental information (positional information, timeinformation, etc.). That is, in the case of presenting contentinformation, content information suitable for a viewing environment canbe preferentially presented.

3. Third Embodiment Information Presentation System

FIG. 11 illustrates an example of the configuration of an informationpresentation system 300 according to a third embodiment of the presenttechnology. The information presentation system 300 includes an objectrecognizing unit 301, an object identification database 302, a gesturerecognizing unit 303, a gesture identification database 304, aninformation presenting unit 305, and a presentation information database306.

The object recognizing unit 301 performs processing for detecting andidentifying an object for image data and obtains object information, onthe basis of object identification data stored in the objectidentification database 302. The image data used here may be image dataobtained by capturing an image with an image pickup unit (camera) orimage data stored in a semiconductor memory, a hard disk drive, or thelike serving as a storing unit. In this case, the common objectidentification database 302 is used in a system so that an object is notrecognized as a different object. Here, objects include human beings andanimals.

An object state acquired by the object recognizing unit 301 includes,for example, object identification information (object ID) andpositional information on a camera (image pickup unit) relative to theobject. The camera positional information is based on, for example, theposition of the object, and serves as relative positional information onthe relative position between the object and the camera. The objectrecognizing unit 301 recognizes an object on the basis of, for example,positions of characteristic points of the object contained in an image.

The object recognizing unit 301 also calculates, on the basis of thepositions of the characteristic points of the object, for example, theposition of the camera (image pickup unit) relative to the object. Thatis, the object recognizing unit 301 calculates the position of thecamera on the basis of how the object (target) appears in the image andon the basis of the actual size of the object described in a recognitiondictionary registered in advance in the object identification database302. The processing by the object recognizing unit 301 will not beexplained in detail below and is described, for example, in JapaneseUnexamined Patent Application Publication No. 2008-304268 by thisapplicant.

The gesture recognizing unit 303 performs processing for detecting andidentifying a gesture for the camera positional information output fromthe object recognizing unit 301 and obtains gesture identificationinformation (gesture ID), on the basis of gesture identification datastored in the gesture identification database 304. The gestureidentification information includes certainty information added thereto,representing the degree of certainty. That is, the gesture recognizingunit 303 compares the position or track of the camera positionalinformation with gesture identification data, and calculates and outputsthe closest gesture identification information and certainty informationof the gesture identification.

The information presenting unit 305 acquires content information, suchas images, sound, or text, associated with object identificationinformation and gesture identification information, on the basis of theobject identification information, the gesture identificationinformation, and certainty information, and presents the acquiredcontent information. In this processing, if the degree of certainty ofgesture identification is smaller than or equal to a specific value, theinformation presenting unit 305 presents a plurality of pieces ofinformation associated with the object identification information andwaits to receive input of a gesture. Gesture recognition performed heremay be replaced by touch point or button operation and may be apartially or uniformly hybrid operation.

The information presentation system 300 illustrated in FIG. 11 can alsobe, for example, the client device 500A illustrated in FIG. 2A or thenetwork system illustrated in FIG. 2B including the client device 500Aand the server device 500B that are connected over the network 500C. Inthe case of the network system illustrated in FIG. 2B, units forming theinformation presentation system 300 are appropriately allocated in adistributed manner to the client device 500A and the server device 500B.For example, the object recognizing unit 301, the object identificationdatabase 302, the gesture recognizing unit 303, the gestureidentification database 304, and the presentation information database306 can be allocated to the server device 500B.

An example of a process for performing information presentation by theinformation presentation system 300 illustrated in FIG. 11 will now beexplained with reference to a flowchart illustrated in FIG. 12. In stepST31, the information presentation system 300 starts the process, forexample, when the power is turned on. Then, the information presentationsystem 300 proceeds to processing of step ST32.

In step ST32, the object recognizing unit 301 performs processing fordetecting and identifying an object for input image data and obtainsobject identification information. In this processing, for example, auser captures an image of a specific object using an image pickup unit(camera) attached to the client device 500A to obtain input image data.Alternatively, in this processing, for example, a user selects imagedata stored in a storing unit of the client device 500A to obtain inputimage data.

In step ST33, the information presentation system 300 determines whetheror not an object has been identified. When an object has beenidentified, the gesture recognizing unit 303 determines whether or not agesture has been identified in step ST34. In this processing, when thedegree of certainty of gesture identification is greater than a specificvalue, the information presentation system 300 determines that a gesturehas been identified.

When a gesture has been identified, the information presenting unit 305acquires content information, such as images, sound, or text, associatedwith the object identification information and gesture identificationinformation from the presentation information database 306, and presentsthe acquired content information in step ST35. When a gesture has notbeen identified, the information presenting unit 305 presents aplurality of pieces of content information, such as images, sound, ortext, associated with the object identification information in stepST36, and waits to receive input of a gesture.

An example of a method for recognizing a gesture will now be explained.Here, gesture recognition based on position will be explained. In thiscase, as illustrated in FIG. 13, threshold processing is performed forthe coordinates (X0, Y0, Z0) of the camera position represented bycamera positional information acquired by the object recognizing unit301. That is, it is calculated whether or not the camera position fallswithin a specified space. In this processing, any type of coordinatesystem can be used. In FIG. 13, examples of specified spaces areexpressed by broken lines. The individual specified spaces may be incontact with each other.

In the case where moving image data is input, the camera position can beobtained in a time sequence manner. Thus, in this case, it is importantto ensure the stability in operations performed by a user by properlyperforming filtering using a median filter or the like. Furthermore, thestate where a recognized object does not exist in a frame, that is,object recognition is not performed, is detected as a state and theresult is output.

Various other methods, such as recognition based on time-sequenceprocessing for tracks (Hidden Markov Model (HMM), Dynamic Programming(DP) matching, etc.) and a method in which track-following time isapplied using a particle filter or the like, can be employed. However,any method can be employed as long as it is a method for recognizing agesture on the basis of only the relative positional relationshipbetween an object (target) and a camera.

Specific Examples of Implementation

Specific examples of implementation will now be explained. Here, anexample of an information viewing system and an example of an iteminformation presentation system will be explained as a first example anda second example, respectively.

First Example Information Viewing System

In augmented reality, for example, when information is acquired byholding a camera over a calendar or a poster on a wall, a billboard on aroad, an insert in newspaper, a membership card for a golf drivingrange, or the like, a plurality of pieces of information are presented.A user wants to perform operations such as selection, enlargement,cancellation, and the like for the presented information. With the useof the information presentation system 300 illustrated in FIG. 11, theseoperations can be realized.

In this example, an information terminal such as a cellular phoneincluding a camera (image pickup unit) is assumed. For example, whencapturing an image of a target, a user is able to receive thepresentation of a plurality of types of information that can beselected, as illustrated in part (a) of FIG. 14. Then, as illustrated inpart (b) of FIG. 14, when the camera is translated, the selected andfocused position can be changed. At this time, for the convenience ofuser operation, for example, a guide representing the selected andfocused position is indicated, and the guide is moved so as to representthe selected and focused position in accordance with the movement of thecamera position.

Then, as illustrated in part (c) of FIG. 14, when the camera is movedcloser to the target while a type of information is being selected, thetype of information can be selected, and the first piece of the selectedtype of information is presented. In this state, as illustrated in part(d) of FIG. 14, when the camera is moved so as to form a circle abovethe target, the next piece of information is presented. In this state,registration of information, editing of registered information, and thelike can also be performed.

When the camera in the state where a certain type of information isselected as illustrated in part (a) of FIG. 15 is moved away from thetarget as illustrated in part (b) of FIG. 15, the state where thecertain type of information is selected can be cancelled. In this state,when the camera is moved so as to form a triangle above the target asillustrated in part (c) of FIG. 15, information registered in the typeof information is deleted.

Second Example Item Information Presentation System

An example in which a target is moved while a camera is being fixed,which is contrary to the first example described above, will beexplained as a second example. In augmented reality, information can beacquired when a user holds a camera over a target. For example, asillustrated in FIG. 16, an image of the back side of transparent shelvesis captured with a camera. Information to be presented is changed inaccordance with which part of the shelves an item is placed by a user.This system can be implemented without difficulty since the system doesnot use an identification chip or the like and the system uses only acamera and a monitor.

FIG. 17 illustrates an example of the configuration of an informationregistration system 400 for registering content information such asimages, sound, and text in the presentation information database 306 ofthe information presentation system 300 illustrated in FIG. 11. Theinformation registration system 400 includes an object recognizing unit401, an object identification database 402, a gesture recognizing unit403, a gesture identification database 404, and an informationregistering unit 405. The information registration system 400 alsoincludes an information input unit 406, a presenting unit 407, anoperation input unit 408, and a presentation information database 306.

The object recognizing unit 401 is configured similarly to the objectrecognizing unit 301 of the information presentation system 300illustrated in FIG. 11. The object recognizing unit 401 performsprocessing for detecting and identifying an object for image data andobtains object information, on the basis of object identification datastored in the object identification database 402. The image data usedhere may be image data obtained by capturing an image with an imagepickup unit (camera) or image data stored in a semiconductor memory or ahard disk drive serving as a storing unit. In this case, the commonobject identification database 402 is used in a system so that an objectis not recognized as a different object. Here, objects include humanbeings and animals.

An object state acquired by the object recognizing unit 401 includes,for example, object identification information (object ID) andpositional information on a camera (image pickup unit) relative to theobject. The camera positional information is based on, for example, theposition of the object, and serves as relative positional information onthe relative position between the object and the camera. The objectrecognizing unit 401 recognizes an object on the basis of, for example,positions of characteristic points of the object contained in an image.

The object recognizing unit 401 also calculates, on the basis of thepositions of the characteristic points of the object, for example, theposition of the camera (image pickup unit) relative to the object. Thatis, the object recognizing unit 401 calculates the position of thecamera on the basis of how the object (target) appears in the image andon the basis of the actual size of the object described in a recognitiondictionary registered in advance in the object identification database402. The processing by the object recognizing unit 401 will not beexplained in detail below and is described, for example, in JapaneseUnexamined Patent Application Publication No. 2008-304268 by thisapplicant.

The gesture recognizing unit 403 is configured similarly to the gesturerecognizing unit 303 of the information presentation system 300illustrated in FIG. 11. The gesture recognizing unit 403 performsprocessing for detecting and identifying a gesture for the camerapositional information output from the object recognizing unit 401 andobtains gesture identification information (gesture ID), on the basis ofgesture identification data stored in the gesture identificationdatabase 404.

The information registering unit 405 registers content information inputfrom the information input unit 406, in association with the objectidentification information (object ID) and the gesture identificationinformation (gesture ID) described above, in the presentationinformation database 306. The content information may be, for example,various types of content information such as images, sound, and text.The information registering unit 405 registers a type of contentinformation that is selected by a user, for example, in accordance withan operation performed by the user on the operation input unit 408 usinga GUI displayed on the presenting unit 407.

The information registration system 400 illustrated in FIG. 17 can alsobe, for example, the client device 500A illustrated in FIG. 2A or thenetwork system illustrated in FIG. 2B including the client device 500Aand the server device 500B that are connected over the network 500C. Inthe case of the network system illustrated in FIG. 2B, units forming theinformation registration system 400 are appropriately allocated in adistributed manner to the client device 500A and the server device 500B.For example, the object recognizing unit 401, the object identificationdatabase 402, the gesture recognizing unit 403, the gestureidentification database 404, and the presentation information database306 can be allocated to the server device 500B.

An example of a process for performing information registration by theinformation registration system 400 illustrated in FIG. 17 will now beexplained with reference to a flowchart illustrated in FIG. 18. In stepST41, the information registration system 400 starts the process. Then,the information registration system 400 proceeds to processing of stepST42. In step ST42, the object recognizing unit 401 performs processingfor detecting and identifying an object for input image data, andobtains object identification information. In this processing, forexample, a user captures an image of a specific object with an imagepickup unit (camera) attached to the client device 500A, and obtainsinput image data. Alternatively, for example, the user selects imagedata stored in a storing unit of the client device 500A, and obtainsinput image data. The information registration system 400 determineswhether or not an object has been identified in step ST43. When anobject has been identified in step ST43, the process proceeds to stepST44.

In step ST44, the information registration system 400 determines whetheror not inputting for information registration has been performed. Whenthe gesture recognizing unit 403 identifies a gesture or a registrationbutton of the operation input unit 408 is pressed, the informationregistration system 400 determines that inputting for informationregistration has been performed. When inputting for informationregistration has been performed, the information registration system 400proceeds to step ST45.

In step ST45, the information registration system 400 presentsinformation registration GUI based on object identification information(object ID) and gesture identification information (gesture ID), on thepresenting unit 407. That is, the object identification information, thegesture identification information, and a type of content informationthat can be input are presented on the presenting unit 407. For example,when any of images, sound, and text can be input, a message indicatingthat any of images, sound, and text can be input is displayed on thepresenting unit 407.

In step ST46, the information registration system 400 determines whetheror not the presented type of content information has been input. Theuser is able to input desired content information from the informationinput unit 406. When content information is input, the informationregistering unit 405 registers the input details in the presentationinformation database 306, in association with the object identificationinformation and the gesture identification information.

As described above, in the information presentation system 300illustrated in FIG. 11, presentation of information is controlled on thebasis of object identification information and relative positionalinformation (gesture identification information). Thus, informationassociated with an object and a gesture can be fetched and presentedfrom the presentation information database 306. Furthermore, informationdesired by a user can be presented without a touch point or buttonoperation. For example, when capturing an image of a target with acamera (image pickup unit), a user is able to perform an operation onlyby moving a camera or moving a target. Consequently, a plurality ofpieces of information can be presented for a target (object).

4. Modifications

In the embodiments described above, individual databases are arrangedseparately. However, a database may include a plurality of types ofdatabases. Furthermore, in the embodiments described above, in the casewhere a network system includes individual devices, individual units areappropriately allocated to the client device 500A and the server device500B in a distributed manner.

However, the client device 500A may include all the functional units,and the server device 500B may include a recognizing unit and adatabase, in an appropriate manner. In this case, when communication issatisfactory, the server device 500B can efficiently perform recognitionprocessing. Even when communication is not satisfactory, all theprocessing can be performed only by the client device 500A, thus causingno trouble in presentation of information.

Furthermore, in the embodiments described above, object recognition isperformed in an optical manner (using a camera etc.). However, objectrecognition is not necessarily performed in an optical manner (using acamera etc.). For example, any unit, such as a wireless device(Bluetooth® etc.) or a combination of a distance surveying device(ultrasonic, laser, etc.) and a sensor having an identifying function,can be used as long as the unit can identify an object and acquire therelative position.

The present technology may employ the following configurations:

(1) An information registration device including an object informationacquiring unit that acquires object information obtained by performingprocessing for detecting and identifying an object for image data; and acontent information registering unit that registers target contentinformation in a database in association with the object information.

(2) The information registration device described in (1), wherein theobject information includes meta-information on the object as well asobject identification information.

(3) The information registration device described in (2), wherein themeta-information includes at least one of positional information on theobject and attitude information on the object.

(4) The information registration device described in any one of (1) to(3), further including a user identification information acquiring unitthat acquires user identification information obtained by processing fordetecting and identifying a user, wherein the content informationregistering unit registers the target content information in thedatabase in association with the object information and the useridentification information.

(5) The information registration device described in any one of (1) to(4), further including a registration environmental informationacquiring unit that acquires registration environmental information,wherein the content information registering unit registers the targetcontent information in the database in association with the objectinformation and the registration environmental information.

(6) The information registration device described in (5), wherein theregistration environmental information includes at least one ofpositional information and time information.

(7) An information registration method including acquiring objectinformation obtained by performing processing for detecting andidentifying an object for image data; and registering target contentinformation in a database in association with the object information.

(8) A program for causing a computer to function as object informationacquiring means for acquiring object information obtained by performingprocessing for detecting and identifying an object for image data; andcontent information registering means for registering target contentinformation in a database in association with the object information.

(9) An information registration system including an image data acquiringunit that acquires image data; an object recognizing unit that acquiresobject information by performing processing for detecting andidentifying an object for the image data; a database; a contentinformation input unit that inputs target content information in thedatabase; and a content information registering unit that registers thetarget content information in the database in association with theobject information.

(10) An information presentation device including an object informationacquiring unit that acquires object information obtained by performingprocessing for detecting and identifying an object for image data; and acontent information presenting unit that acquires content informationassociated with the object information and presents the acquired contentinformation.

(11) The information presentation device described in (10), wherein theobject information includes meta-information on the object as well asobject identification information.

(12) The information presentation device described in (11), wherein themeta-information includes at least one of positional information on theobject and attitude information on the object.

(13) The information presentation device described in any one of (10) to(12), further including a user identification information acquiring unitthat acquires user identification information obtained by processing fordetecting and identifying a user, wherein the content informationpresenting unit acquires content information associated with the objectinformation and the user identification information and presents theacquired content information.

(14) The information presentation device described in any one of (10) to(13), further including a viewing environmental information acquiringunit that acquires viewing environmental information, wherein thecontent information acquired by the content information presenting unitincludes registration environmental information added thereto, and thecontent information presenting unit determines the order in which theacquired content information is to be presented, the content informationincluding a specific number of pieces of information, on the basis ofthe viewing environmental information and the registration environmentalinformation.

(15) The information presentation device described in any one of (10) to(14), further including an image pickup unit that acquires the imagedata; and a relative positional information acquiring unit that acquiresrelative positional information on the relative position between theobject and the image pickup unit, the relative positional informationbeing acquired by performing processing on the image data, wherein thecontent information presenting unit acquires content informationassociated with the relative positional information as well as theobject information and presents the acquired content information.

(16) An information presentation method including acquiring objectinformation obtained by performing processing for detecting andidentifying an object for image data; and acquiring content informationassociated with the object information and presenting the acquiredcontent information.

(17) A program for causing a computer to function as object informationacquiring means for acquiring object information obtained by performingprocessing for detecting and identifying an object for image data; andcontent information presenting means for acquiring content informationassociated with the object information and presenting the acquiredcontent information.

(18) An information presentation system including an image dataacquiring unit that acquires image data; an object recognizing unit thatacquires object information by performing processing for detecting andidentifying an object for the image data; and a content informationpresenting unit that acquires content information associated with theobject information and presents the acquired content information.

The present disclosure contains subject matter related to that disclosedin Japanese Priority Patent Application JP 2011-249178 filed in theJapan Patent Office on Nov. 14, 2011, the entire contents of which arehereby incorporated by reference.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

What is claimed is:
 1. An information registration device comprising: aprocessor configured to: acquire object information obtained byperforming processing for detecting and identifying an object for imagedata; acquire relative position information regarding a relativeposition of a camera with respect to the object; register a plurality ofpieces of target content information in a database in association withthe object information; and control an order in which the plurality ofpieces of target content information are presented on a display based ona priority determined for each of the plurality of pieces of targetcontent information, wherein the priority for each of the plurality ofpieces of target content information is determined based on at least therelative position information regarding the relative position of thecamera with respect to the object.
 2. An information registration methodperformed by an electronic device, the method comprising: acquiring, bythe electronic device, object information obtained by performingprocessing for detecting and identifying an object for image data;acquiring, by the electronic device, relative position informationregarding a relative position of a camera with respect to the object;registering, by the electronic device, a plurality of pieces of targetcontent information in a database in association with the objectinformation; and controlling, by the electronic device, an order inwhich the plurality of pieces of target content information arepresented on a display based on a priority determined for each of theplurality of pieces of target content information, wherein the priorityfor each of the plurality of pieces of target content information isdetermined based on at least the relative position information regardingthe relative position of the camera with respect to the object.
 3. Anon-transitory computer-readable storage medium storingcomputer-readable instructions, that when executed by a computer, causethe computer to execute a method comprising: acquiring objectinformation obtained by performing processing for detecting andidentifying an object for image data; acquiring relative positioninformation regarding a relative position of a camera with respect tothe object; registering a plurality of pieces of target contentinformation in a database in association with the object information;and controlling an order in which the plurality of pieces of targetcontent information are presented on a display based on a prioritydetermined for each of the plurality of pieces of target contentinformation, wherein the priority for each of the plurality of pieces oftarget content information is determined based on at least the relativeposition information regarding the relative position of the camera withrespect to the object.
 4. The non-transitory computer-readable storagemedium according to claim 3, wherein the object information includesmeta-information on the object as well as object identificationinformation.
 5. The non-transitory computer-readable storage mediumaccording to claim 4, wherein the meta-information includes at least oneof positional information on the object and attitude information on theobject.
 6. The non-transitory computer-readable storage medium accordingto claim 3, further comprising: acquiring user identificationinformation obtained by processing for detecting and identifying a user;and registering the plurality of pieces of target content information inthe database in association with the object information and the useridentification information.
 7. The non-transitory computer-readablestorage medium according to claim 2, further comprising: acquiringregistration environmental information; and registering the plurality ofpieces of target content information in the database in association withthe object information and the registration environmental information.8. The non-transitory computer-readable storage medium according toclaim 7, wherein the registration environmental information includes atleast one of positional information and time information.
 9. Aninformation registration system comprising: a database; and a processorconfigured to: acquire image data; acquire object information byperforming processing for detecting and identifying an object for theimage data; acquire relative position information regarding a relativeposition of a camera with respect to the object; input a plurality ofpieces of target content information in the database; register theplurality of pieces of target content information in the database inassociation with the object information; and control an order in whichthe plurality of pieces of target content information are presented on adisplay based on a priority determined for each of the plurality ofpieces of target content information, wherein the priority for each ofthe plurality of pieces of target content information is determinedbased on at least the relative position information regarding therelative position of the camera with respect to the object.
 10. Aninformation presentation device comprising: a processor configured to:acquire object information obtained by performing processing fordetecting and identifying an object for image data; acquire relativeposition information regarding a relative position of a camera withrespect to the object; acquire a plurality of pieces of contentinformation associated with the object information and present theacquired plurality of pieces of content information; and control anorder in which the plurality of pieces of content information arepresented on a display based on a priority determined for each of theplurality of pieces of content information, wherein the priority foreach of the plurality of pieces of content information is determinedbased on at least the relative position information regarding therelative position of the camera with respect to the object.
 11. Aninformation presentation method performed by an electronic device, themethod comprising: acquiring, by the electronic device, objectinformation obtained by performing processing for detecting andidentifying an object for image data; acquiring, by the electronicdevice, relative position information regarding a relative position of acamera with respect to the object; acquiring a plurality of pieces ofcontent information associated with the object information andpresenting the acquired plurality of pieces of content information; andcontrolling, by the electronic device, an order in which the pluralityof pieces of content information are presented on a display based on apriority determined for each of the plurality of pieces of contentinformation, wherein the priority for each of the plurality of pieces ofcontent information is determined based on at least the relativeposition information regarding the relative position of the camera withrespect to the object.
 12. A non-transitory computer-readable storagemedium storing computer-readable instructions, that when executed bycomputer, cause the computer to execute a method comprising: acquiringobject information obtained by performing processing for detecting andidentifying an object for image data; acquiring relative positioninformation regarding a relative position of a camera with respect tothe object; acquiring a plurality of pieces of content informationassociated with the object information and presenting the acquiredplurality of pieces of content information; and controlling an order inwhich the plurality of pieces of content information are presented on adisplay based on a priority determined for each of the plurality ofpieces of content information, wherein the priority for each of theplurality of pieces of content information is determined based on atleast the relative position information regarding the relative positionof the camera with respect to the object.
 13. The non-transitorycomputer-readable storage medium according to claim 12, wherein theobject information includes meta-information on the object as well asobject identification information.
 14. The non-transitorycomputer-readable storage medium according to claim 13, wherein themeta-information includes at least one of positional information on theobject and attitude information on the object.
 15. The non-transitorycomputer-readable storage medium according to claim 12, furthercomprising: acquiring user identification information obtained byprocessing for detecting and identifying a user; and acquiring theplurality of pieces of content information associated with the objectinformation and the user identification information and presenting theacquired plurality of pieces of content information.
 16. Thenon-transitory computer-readable storage medium according to claim 12,further comprising: acquiring viewing environmental information, whereinthe acquired plurality of pieces of content information includesregistration environmental information added thereto, and the order inwhich the acquired plurality of pieces of content information is to bepresented, the plurality of pieces of content information including aspecific number of pieces of information, is determined on the basis ofthe viewing environmental information and the registration environmentalinformation.
 17. The non-transitory computer-readable storage mediumaccording to claim 12, further comprising: acquiring the image data,wherein the relative position information is acquired by performingprocessing on the image data, and the plurality of pieces of contentinformation associated with the relative position information as well asthe object information are acquired and the acquired plurality of piecesof content information are presented.
 18. The non-transitorycomputer-readable storage medium according to claim 12, furthercomprising: selecting, from among the plurality of pieces of contentinformation, a first piece of content information to be presented on thedisplay based on the relative position information regarding therelative position of the camera with respect to the object.
 19. Thenon-transitory computer-readable storage medium according to claim 18,wherein a different piece of content information, from among theplurality of pieces of content information, is selected to be presentedbased on a different relative position of the camera with respect to theobject.
 20. An information presentation system comprising: a processorconfigured to: acquire image data; acquire object information byperforming processing for detecting and identifying an object for theimage data; acquire relative position information regarding a relativeposition of a camera with respect to the object; acquire a plurality ofpieces of content information associated with the object information andpresent the acquired plurality of pieces of content information; andcontrol an order in which the plurality of pieces of content informationare presented on a display based on a priority determined for each ofthe plurality of pieces of content information, wherein the priority foreach of the plurality of pieces of target content information isdetermined based on at least the relative position information regardingthe relative position of the camera with respect to the object.